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ABSTRACT 

A criterion system based on client perspective was 
developed to appraise the effects of counseling. There were four 
subject samples: (1) 1U2 clients seen for individual counselxng 
during 1967-58; (2) 42 clients counseled during 1969-70; (3) 17 
clients seen for group counseling; and (4) 60 respondents to a letter 
asking for volunteers. The instruments used were a goals checklist 
which contained a broad range of personal goals, and a follow-up 
questionnaire mailed to each subject. Clients marked their goals on 
the checklist, were counseled, and later reported by the 
questionnaire significant events indicating progress toward thexr 
goals. Inter judge agreement on classification of the questionnaire 
responses into 29 categories exceeded 85%. Goalb and questionnaire 
categories ranged from general attitudes to specific iehaviors. The 
system is discussed in terms of its potential usefu7.ness, validity, 
and versatility. (Author/BW) 



Stcpc* Toward Outcojne Criteria in Couni;ol inj?; and rL^vchothevopy 
Andrew Thompson an'd Ad(un I-iiller 

Two of the difficulties facing any r(?Gcarcher \:liO intends to conduct a 
follow-up atud^Y on the outconos cf counncling and psychotherapy arc to de- 
cide *'vho" should be the j\id(^e of success, and "what*' criteria constitute 
success. The jud^e could be the researclier, the therapist, or the client, 
llie viewpoints of the various judges do not necessarily converge; in fact 
as Strupp and Bergin (19^9) note, client self-evaluation of dof,ree of suc- 
cess usually falls out as a separate factor in factor analytic st\idies 
(Cartwri/rjit , Robertson, Fisl;e, & Kirtner, I96I*, Forsyth & Fair^cather, I961). 
But perhaps even more important, the client may not af^ree with others as to 
what would be an appropriate measure of success for him. Thus, for example, 
the researcher may decide that raising a student's GPA is a good thing (e.g., 
Hill & Grienecks, I966) or increasing his length of employment (e.g., Zax & 

Rickles, V/eise, Gray, & Yee , I966) or getting a ^'better" profile on innumer- 
able other psychologiceJL inventories. In each case the client-subject may, 
if asked, select other potential changes in his life av- being much more sxFy- 
nificemt and relevant to his particular goals. 

One way of circvunventing this limitation has been for the therapist 
and the client to jointly decide on some target behaviors at the beginning 
of treatment. However*, it is quite possible that the client will find it 
difficult and artificial to specify such behaviors and will do so only for 
the sake of the research and/or to please his therapist'. He may consider 
the target behaviors thus arrived at substantially peripheral to his real 
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concerns. And finally, it is likely that h^j crul/or hin tliorapist^y /^oals 
for the trcatrnent will change durln^T; the course of counseling or therapy 
(Thompson & Zimir.ermann , 1969 ). 

For tliese reasons it was decided to try to develop a criterion system 
based primarily on the client's point of view, with provisions for ony chanpo 
in that perspective during counseling;; or therapy. It was hoped that this 
system would be broad enough and varied enouf^h to fit u variety of treatments 
and that the measurement procedures used for the criterion system would be 
reliable, practically useful, and easily validated. 

Subjects 

There were four subject samples. The first consisted of lU2 clients 
who were seen at the University of Oregon counseling center for indivlduo?. 

^ • * t i/N/'rti •...•^ ir^/'O mi ... .... I.J. - .1 .... 

questionnaires from 10 to 19 months after the termination of counseling. 
The second sample consisted of a similar group of '♦2 clients who had ini- 
tiated counseling during the 1969-70 academic year. They had not returned 
for individual counseling from two to ten months prior to December, 1970, at 
which time the follow-up questionnaire was mailed. 

The third sample consisted of IT clients seen for group counseling or 
therapy, Tliey were from five groups offered during Fall term, 19T0, and 
received follow-up questionnaires seven to eight w<^eks later. The fourth 
sample consisted of 6o respondents to a letter sent in early 19T1 asking for 
volunteers from 211 nonclient students randomly selected from the university 
population. They were sent follow-up questionnaires two to three months 
after completing the initial goal checklist. 
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Tlie percentage of UGoablo returns from tlie four sampler ranp;ed frcin 
r}Qp (nonclients) to 1*5)^ (croup members). Ko attempt vac made to obtain a 
greater rate of return since the purpose of sornpling was to obtain a diver- 
sity of input, and it was thou{i;ht that sufficient diversity vms achieved by 
samplin(5 ticross a relatively vide range of counselor-client combinations 
and follow -up periods. 

Instruments and P rocedure ?; 

-There were two basic instruments used, each of which had been developed 
over a period of years and which had undergone several revisions. The 
first was a goal checklist which contained a broad range of personal goals 
that had been found to be of frequent concern to clients at the University 
of Oregon counseling center, and which included such goals as "more sensi- 
tivity and perceptiveness in social situations", "determination of careers 
for which I am best suited", "fewer, less intense periods of depression", 
and "loss of weight". The I96T-68 checklist had 37 goals; the 1970 check- 
list was slightly revised and contained 38 goals. A fuller description of 
the checklist, its rationale, and its uses can be found in a previous article 
(Thompson & Zimmermann, I969). 

Clients were instructed to check any goal of Qoncem to them that they 
wished their counselor to help them obtain. The nonclient sample were 
asked to check goals "that really are personally meaningful to you and vhich 
you hope to make progress towards in the near future;" The clients were 
given the checklist to complete before they were seen for their first appoint- 
ment or their first group meeting. Nonclient volunteers completed the check- 
list in response to the letter asking their participation in the study ♦ 
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The second bucie instrumont was the follow-up quef;tionnaii*e that was 
mailed to eaeh aubjc-et. Moiled alonp; with the questionnaire was a fie- 
titious example of a completod questionnaire, illustrating the dof^ree of 
detail appropriate for responding. Also enclosed was a eompleted copy of 
the subject's own r>oal checklist. 

The questionnaire consisted of four parts. In the first part tho sub- 
ject v/as asked to rate the dcGree of progress he had made towards his goals. 
Each goal was assicner to one of the following four cate/];ories : '^definite 
progress", "some progress", "no progress", and "doing worse," Secondly, he 
was asked to "indicate what sorts of events have occurred which support the 
progress ratings you assigned above," Thirdly, he was asked if there were 
any other goals that he did not mark initially but which proved to be relevant 
later, and if so, to indicate which goals these were and what prof^ress he 
had made toward them. Finally, he was asked whether his counselor was of any , 
aid in achieving any of the goals he had markr^d or considered relevant. Only 
the data provided by parts two and three will be reported in this study, since 
it is the responses to these parts that constitute the clients* perceptions 
of what progress meant for themi 

RESULTS 

Initially 73 categories were devised into whi-ch the variety of events 
reported in parts two and three of the follow-up questionnaire could be clas- 
sified. However, by eliminating or combining infrequently used categories, 
the number was reduced to 29 for the 1969-70 subjects and to 28 for the 1967- 
68 sample. The 29 categories are listed in Table 1. Those considered specific 
enough to qualify as target behaviors arc asterisked, 
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Intcrjude e Apjreement 

The cateGorizaticn procedure required that each Jud^e separiitcly cr.te- 
Rori-ze the events for each quentionnaire. A categoriviation form v:xv. developed 
for aase of classification and for convenience in checking, judpe ncrceir.ont. 
In preparation for use of the scheme, each judne studied the written descrip- 
tions of each category. Then each .ludge sepurr^tely classified reports for 
a small representative number of questionnaires. Finally, Judges discussed 
and reconciled discrepancies. 

The two authors categorized 75 of the 26l follov-up questionnaires in 
common. Each of the four subject samples were represented. ITieir overall 
inter Judce agreement was 88.97*. The lead author and a doctoral intern cate- 
gorized another diverse sampling of 20 questionnaires and obtained 
agreement. This was computed by dividing the total nuinber of responses (dis- 
agreements were weighted double) into the number of responses that were class 
ified in the same categories by both Judges. 

Insei't Table 1 abo\it here 

— — — — "~ "~ "~ ""^ 

In order to determine the extent to which subjects voluntarily report 
specific changes as opposed to broad general ones each sample was mialyzed 
in terms of the percentage of the total responses that occurred in specific 
categories lending themselves to target behaviors (asterisked in Table l) . 
It was decided to exclude category one, "changes in overall attitude towards 
self", from this analysis since the instructions for using this category 
practically guaranteed that it would be checked. It was to btJ checked any- 
time there v. : a preponderance of either positive or negative changes in any 
of the other categories. 
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Forty^oi(;ht 3)erccnt of the resporiises of the 1967-68 individual client 
samples verc classified into specific cciteconen* The correspond inf^ percent- 
ages for the other samples vere 3^% for the 1969-70 individual client siunple, 
Uh% for the nonclients and for the r.roup clients • The lonner time before 
follow-up of the earlier individual client sair.ple (10 to 19 nionths contrasted 
with 2 to 10 months) may have promoted r.reatcr use of the catcf;ories having 
to do with decisions rcf-arding acadendc r.ajors> careers, aiid marital status, 
all of which are specific behaviors. The nonclient sample chose a greater 
proportion of specific vocationa3.-acadeniic goals on their initial, checklists 
than any of the other three sampler, jxnd gave correspondingly more responses 
in the specific follow-up categories in these ai-eas. The group sample chose 
a greater proportion on interpersonal goals and gave corr.:ispondingly more 
responses in generval categories 23 and 26 having to do with general changes 
in interpersonal relations . 

Fifteen percent of the total subject sample gave responses that were 
classified in the final miscellaneous category, indicating that the preceding 
28 categories were reasonably but not completely inclusive of the follow-up 
events that were considered relevant by the subjects. 

DISCUSSION 

The primary goal of this study was'to develop and evaluate a criterion 
system for counseling or psychotherapy based on client perspective of what is 
relevant, rather than on criteria advanced by therapists, researchers, or 
others. The intention was not to supplant other criterion systems, but to 
complement them, thus increasing the number and type of potentially useful 
and relevant criteria for follovr-up research. Krumboltz (1966) and Paul (196?) 
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have arnued that need to particular! -/.e our treatv.ont and outcome niODSurcs 
to fit each client and each therapist anO/or treatment modality. But it is 
difficult, and perhaps impossible to make such particularizations if we do 
not have a clear picture of what the clients themselves think is relevtir.t 
in terms of outcomes . 

There was nothinc surprising about the categories that evolved from 
analysis cf the events and feelings that clients thouRht were relevant to 
their. experience of progress (or lack of progress). All of these things 
have probably been parts of previous therapist- or researcher-developed cri- 
teria of success. However, there are many ways to cut a cake and this cate- 
gory scheme has at least the advantages of being: l) certified by the clients 
to a greater extent than most systems; and 2) reliable in terms of agreement 
of classification by Judges; and 3) apparently inclusive of most criteria 
that clients from a university population deem relevant in terms of counseling 
and psychotherapy . Also many of the outcomes were quite specific and would 
require little checking to determine if they actually occurred. Others could 
be readily ascertained by a variety of current psychological tests which re- 
flect changes in mood or self concept. 

Further steps in utilization of this system might involve development of 
a questionnaire or checklist based on the 28 categories which clients could 
complete either during 'or following counseling'or therapy, or both. Part of , 
this questionnaire wo.ad consist of an estimation by the client of the relevancy 
of each of the .outcome categories to his. present concerns. The data thus 
acquired should provide a composite description of the client's progress. 
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TABLE 1 

Catecories for Questionnaire Rci^ponscs 



1. Chanf,cs in overall attitude toward self. 

2. Changes in overall scnr^c of direction. 

3. Changes in feelinr,s about self maturity, compctenee and independence. 

14. Changes in level of dcpreycion • 

5, Changes in level of gui].t. 

6, Quit or deeidod to quit sehool.* 

7, Decision re£?:ardinfi; aeadeinie major or pro(!:raii).*^ 

8, Chance in grade point averap;e.'^ 

9, Ch8x»{T,e in level of vork--otudy skills.* 

10. Holding a Job vrhieh is considered relevant to career choice* 

11, Vocational testing.* 

12. Other career exploration (interviews, reading about careers, etc.).* 

13, Chance in level of certainty about vocational future, 

Ik. Change in level of control of specific and named habits.* 

15. Change in level of control of specific but unnamed habits. *-r 

16. Change in marital status.* 

17. Change in level of satisfaction with marital sex.* 

18. Other changes in satisfaction with marital relationship. 

19. Change in satisfactoriness of relations with parental family. 

20. New significant one-to-one relationship.* 

21. Discontinuance or decision to discontinue a significant one-to-one 
relationship.* 

22. Sp'.": ! changes in one-to-one relations (frequency or amount of contact, 
cho.:' >.u level of satisfaction with sexual relations).* 
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TABLE 1 (Cont'd.) 

23. Genera], changes in satisfactorinesG of one-to-one relations. 
2'i. Joined a new croup (other than a therapy Rroup).^ 

25. Change in General level of satisf actoriness of one-to-many relations. 

26. Change in satisf actoriness of interpersonal relations not elsewhere 
classified. 

27. Change in satisf actoriness of level or types of activity. 

28. Change in level of competency in some specific area (has learned new 
skill or has demonstrated increased competency in an old area) .^' 

29. Miscellaneous significant events not elsewhere classified (e.g., moved 
to new location, pregnancy).* 



+ Not included in the 1967-68 sample. 
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